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MEASURING 
What MATTERS 


Does your state collect the data you 
need to answer these questions? 
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wise unaccounted for after 8th grade? 


Which schools produce the strongest 
academic growth for their students? 


What achievement levels in middle school 
indicate that a student is on track to suc- 
ceed in rigorous courses in high school? 


What high school performance indicators 
(e.g., enrollment in rigorous courses or 
performance on state tests) are the best 
predictors of students’ success in college or 
the workplace? 


What percentage of high school graduates 
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A rapidly changing global economy and concerns about our ability to create a 
competitive workforce have focused national attention on the quality of 
America’s education system. The 2005 National Education Summit on High 
Schools, which was sponsored by Achieve and the National Governors 
Association, focused on high schools and their critical role in preparing young 
people for a successful future. Governors and business and education leaders 
at the Summit reviewed sobering data on the gaps each state must close to 
produce better-prepared graduates, and many states have embraced an 
aggressive policy agenda in an effort to close these gaps. 


But gauging the effect of these efforts will be difficult and time consuming if 
States do not have data systems that provide ready access to the high-quality 
information decisionmakers need. 


There is no shortage of data in our education system today. States and 
school districts are gathering large amounts of information on school and 
Student performance. The challenge now is to make sure states are collecting 
the most relevant data and that they have a coordinated system for using 
them effectively. 


Few Students Make It through the Education Pipeline 


For every 100 9th graders ... 


18 graduate 
from college 
on time 


Source: National Center for Public Policy and Higher Education, Policy Alert, 
April 2004. Data are estimates of pipeline progress rather than actual cohort. 
They are not based on longitudinal data. 


THE MANY BENEFITS OF LONGITUDINAL DATA 


To provide educators with the data they need to improve student achievement, states need more than a 
series of one-time snapshots of student performance. They need a system that collects high-quality data 
about how individual students are doing over time, from prekindergarten through 12th grade and into 


postsecondary education. This information — also known as longitudinal data — makes it possible to: 
[> — follow students’ academic progress as they move from grade to grade; 
determine the value-added and efficiencies of specific schools and programs; 


[> — identify consistently high-performing schools so that educators and the public can learn from 
best practices; 


evaluate the effect of teacher preparation and training programs on student achievement; and 


[> focus school systems on preparing a higher percentage of students to succeed in rigorous high 


school courses, college and challenging jobs. 


To take full advantage of the power of longitudinal data, states need systems that can exchange informa- 
tion within and across pre-K—12 and postsecondary systems and states. This will allow states to continue 


monitoring achievement as students move from place to place and through the education pipeline. 


In addition, states need a strategy for training policymakers, educators and others to use this information 


— both to improve policies and practices and to hold schools accountable for achievement gains. 


THE NEED IS URGENT 
Improving student achievement is imperative — our students simply are not learning enough to meet the 
demands of today’s competitive world. Studies show that: 


> Too few students make it through the education system. 


> Jobs that support a middle-class lifestyle and offer opportunities for advancement require higher- 
level skills and more education than ever before. 


Students from other countries routinely outperform U.S. students on international assessments, 
which could signal a threat to our leadership in the global economy. 


Those U.S. students who do graduate high school often require costly remedial education or 
training in basic skills. 


To help make the changes necessary to meet this challenge, educators and policymakers need timely 


access to accurate longitudinal education data. We must work to build the systems at the state level that 
can deliver these data. 
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Aunique statewide student identifier. As students move from grade 
to grade and from district to district, this ID number will allow states to 
accurately measure the progress of every student over time, from pre- 
kindergarten through grade 12. 


Student-level enrollment, demographic and program participation 
information. This information will help identify which programs are 

helping students succeed. It also will help account for students who transfer 
from school to school and ensure that test data are disaggregated correctly. 


The ability to match individual students’ test records from year to 
year to measure academic growth. Being able to match test records 
for individual students from last year to this year will provide valuable 
diagnostic information to teachers and principals and will help educators 
monitor each student's academic growth. 


Information on untested students. With this information, states can 
ensure that students from all groups are participating in state tests and 
account for students who were exempted from the tests. 


A teacher identifier system with the ability to match teachers to 
students. Many states collect data on teacher education and certification, 
but matching teachers to students by classroom and subject is critical to 
understanding the connection between teacher training and qualifications 
and student academic growth. 


Student-level transcript information, induding information on 
courses completed and grades earned. States will be able to track 
course-taking patterns and analyze their relationship to success on state 
assessments and readiness for college and work. 


To build a complete longitudinal data system, states must 
include the following 10 essential elements:* 
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Longitudinal Data System 


Student-level college readiness test scores. Student performance on 
the SAT, SAT II, ACT, Advanced Placement, International Baccalaureate and 


other college readiness exams is a good indicator of whether students are 


states report 
prepared to succeed in postsecondary education and work; however, cur- Eee this 
element 
rently only seven states maintain this information from year to year at the 
student level. But some states are going a step further by building college 


readiness tests into their statewide assessment systems. 


Student-level graduation and dropout rate. A majority of states 
currently collect annual records on individual graduates and dropouts. But 
a new National Governors Association (NGA) compact signed by 48 states 


states report 
aims to create a more valid, reliable and consistent graduation rate that Paes ae 
element 
tracks students from 9th to 12th grade. Based on National Center for 
Educational Accountability analyses, only 14 states currently can calculate 


the graduation rate defined in the NGA compact. 


The ability to match student records between the pre-K-12 and 
postsecondary systems. Opening the lines of communication between 1 ) 


pre-K—12 and higher education is critical to ensuring that students suc- 


states report 
ceed at the postsecondary level. Connecting student performance in col- ee oe 
element 
lege to what happens in high school will give high schools the information 
they need to align curriculum and instruction to ensure that graduates are 


better prepared for college and work. 


A state data audit system assessing data quality, validity and 
reliability. The decisions made in education are only as good as the 1 Q 


information on which they are based. 
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* 1S YOUR STATE COLLECTING THE RIGHT INFORMATION? 
In August 2005, the National Center for Educational Accountability, with support from The Broad 
Foundation and the Bill & Melinda Gates Foundation, conducted a survey about state data systems to 
learn how many states already have the 10 essential elements in place. The results: No state has all 10 
elements, and only eight states have at least seven elements of a longitudinal data system. 


To see state-by-state results and find out more about what it takes to create a longitudinal data system, 
go to www.DataQualityCampaign.org. 


FUTURE DIRECTIONS OF STATE DATA SYSTEMS 


These 10 elements are essential but not sufficient. States need to plan for a series of next-generation improve- 
ments — in fact, some states are already working on them. In the future, data systems can make it possible to: 


> Connect school performance with spending. A longitudinal data system identifies which 
schools and school systems perform well. But to better understand what it costs to improve 
student performance, states also need to collect financial information at the school and 
program levels and link it to individual student achievement data over time. Very few 
states do this today. 


[> Connect school performance to employment and other systems. Educators and 
policymakers need to know whether schools are preparing students for long-term success in 
the workplace, not just in college. Obtaining this information requires matching the academic 
(both pre-K—12 and postsecondary) and employment records of individual students. States 
also should consider incorporating into their education data systems, as needed, records from 
other social service agencies that have information that is relevant to students’ health and safety. 


> Transfer records across systems and states. In an increasingly mobile world, people 
regularly move across state borders, making it difficult to tell, for example, whether a student 
has dropped out or has moved to a new state. Therefore, not only do data systems need to be 
able to exchange information with other systems — such as postsecondary — within the 
state, but they also need to be able to exchange information with systems in other states. The 
key is ensuring that data systems built by different vendors in different states use common 
data standards and definitions. 


CALL TO ACTION 


No state data system currently includes every one of these 10 essential elements. (To see how your state 


stacks up, visit www.DataQualityCampaign.org.) 


All states should make it a priority to put them in place within the next three years. For this reason, a 
group of national organizations has launched the Data Quality Campaign to encourage and support 
policymakers’ efforts to fully develop and use longitudinal data in education. 


States are spending hundreds of millions of dollars to improve student achievement. But without quality 
data, they are essentially flying blind. Policymakers need to act now to put in place the policies and 
resources to ensure that each state has a longitudinal data system and the culture and capacity to trans- 
late the information into specific action steps to improve student achievement. 


This brochure was produced by the Data Quality Campaign and made possible with the financial support 
of the Achieve, Inc., American Diploma Project Network. 


DATA QUALITY CAMPAIGN nA vif 
The Data Quality Campaign is a national, collaborative effort to CAMPAIGN 


Using Data To Improve Student Achievement 


encourage and support state policymakers to: 
> improve the collection, availability and use of high-quality education data and 
> implement state longitudinal data systems to improve student achievement. 


The campaign provides tools and resources that states can use as they develop quality longitudinal data systems 
and also will serve as a national forum for reducing duplication of effort and promoting greater coordination and 


consensus among like-minded organizations. 


Founding partners in the Data Quality Campaign include: 


Achieve, Inc. Alliance for Excellent Education 

Council of Chief State School Officers The Education Trust 

National Center for Educational Accountability National Center for Higher Education Management 
National Governors Association Center for Best Systems 

Practices Schools Interoperability Framework Association 
Standard & Poor's School Evaluation Services State Higher Education Executive Officers 


The campaign is managed by the National Center for Educational Accountability (NCEA) and supported by the Bill & 
Melinda Gates Foundation. For more information, visit the campaign Web site at www.DataQualityCampaign.org. 


AMERICAN DIPLOMA PROJECT NETWORK gi Achieve, Inc. 


Formed out of the 2005 National Education Summit on High Schools, ; : ; 
American Diploma Project Network 

the American Diploma Project (ADP) Network is a coalition of 22 states, 

led by Achieve, Inc., that have committed to align K-12 standards, assessments, curriculum and accountability 


with the demands of postsecondary education and work. ADP states are taking four actions: 
Aligning high school standards with the knowledge and skills required for success after high school. 
Requiring all high school graduates to take challenging courses that prepare them for life after high school. 


Streamlining the assessment system so that the tests students take in high school also can serve as readiness 
tests for college and work. 


Holding high schools accountable for graduating students who are ready for college or careers, and holding 
postsecondary institutions accountable for students’ success once enrolled. 


For more information, visit Achieve’s Web site at www.achieve.org. 


FIND OUT MORE 
Visit the Data Quality Campaign Web site at www.DataQualityCampaign.org for more information about the: 


> 10 essential elements and the state policy actions required to establish, maintain and use a 
quality longitudinal data system; 


results of NCEA’s 2005 survey that show where your state stands on the 10 essential elements; and 


tools, materials, meetings and information that can aid states and interested organizations 
seeking to ensure increased quality, accessibility and use of data. 


